This study explores the integration of Augmented Intelligence (AuI) in Intelligent Tutoring Systems (ITS) to address challenges in Artificial Intelligence in Education (AIED), including teacher involvement, AI reliability, and resource accessibility. We present MathAIde, an ITS that uses computer vision and AI to correct mathematics exercises from student work photos and provide feedback. The system was designed through a collaborative process involving brainstorming with teachers, high-fidelity prototyping, A/B testing, and a real-world case study. Findings emphasize the importance of a teacher-centered, user-driven approach, where AI suggests remediation alternatives while teachers retain decision-making. Results highlight efficiency, usability, and adoption potential in classroom contexts, particularly in resource-limited environments. The study contributes practical insights into designing ITSs that balance user needs and technological feasibility, while advancing AIED research by demonstrating the effectiveness of a mixed-methods, user-centered approach to implementing AuI in educational technologies.

The advent of big data, and of Artificial Intelligence (AI) applications that collect and consume such data, has led to fundamental questions about the ethics of AI designs and to efforts aimed to highlight and safeguard against any potential harms caused by the deployment of AI across diverse domains of applications. Typically, questions raised relate to the trustworthiness of AI as agent technologies that autonomously or semi-autonomously operate in human environments and that have the ability to alter human behaviour. Other questions concern the role that AI may play now and in the future in either resolving or amplifying pre-existing social biases and any resulting harms. Specifically, Ethical AI as an emergent area of AI research and policy, has been spurred by the revelations of AI applications (usually unintentionally) promoting and amplifying many of the discriminatory and oppressive practices, and assumptions that underpin pre-existing social and institutional systems, e.g., historical biases against non-dominant populations, against users characterised by some divergence from the so-called cognitive or physical'norm', or those who are socio-economically disadvantaged (Crawford, 2017a; Madaio et al., 2022; Porayska-Pomsta and Rajendran, 2019; Williamson, Eynon, Knox & Davis, in this volume). Numerous examples of AI bias are both well-documented and rehearsed throughout the emergent ethics of AI literature, in hundreds of policy reports about AI ethics and governance that have been published to date (c.f.

INTRODUCTION With increasing awareness of the societal risks of algorithmic bias and encroaching automation, issues of fairness, accountability, and transparency in data-driven AI systems have received growing academic attention in multiple high-stakes contexts, including healthcare, loan-granting, and hiring (e.g., Barocas & Selbst, 2016; Holstein, Wortman Vaughan, Daumé III, Dudik, & Wallach, 2019; Veale, Van Kleek, & Binns, 2018). Given these noble intentions, why might AIEd systems have inequitable impacts? In this chapter, we ask whether AIEd systems will ultimately serve to A mplify I nequities in Ed ucation, or alternatively, whether they will help to A lleviate existing inequities. We discuss four lenses that can be used to examine how and why AIEd systems risk amplifying existing inequities: (1) factors inherent to the overall socio-technical system design; (2) the use of datasets that reflect historical inequities; (3) factors inherent to the underlying algorithms used to drive machine learning and automated decision-making, and (4) factors that emerge through a complex interplay between automated and human decision-making. Building from these lenses, we then outline possible paths towards more equitable futures for AIEd, while highlighting debates surrounding each proposal. In doing so, we hope to provoke new conversations around the design of equitable AIEd, and to push ongoing conversations in the field forward. PATHWAYS TOWARD INEQUITY IN AIED We begin by presenting four lenses to understand how AIEd systems might amplify existing inequities or even create new ones (cf. While each lens provides a different way of examining pathways towards inequity in AIEd, all are pointed at the same underlying socio-technical system. Figure 1 provides a coarse-grained overview of the broader social-technical systems in which AIEd systems are embedded, and some of the components we will refer to in the four lenses. The accumulated, collective decisions of designers, researchers, policy-makers, and other stakeholders shape these systems' designs. In addition to using or being affected by AIEd systems, on-the-ground stakeholders such as students, teachers, or school administrators may also play a role in shaping their designs; whether directly, through participatory design processes, or indirectly through the passive generation of training data while interacting with an AIEd interface. In turn, decisions regarding what data is used to shape an AIEd system's design (e.g., when used as training data for use with machine learning methods) can shape an AIEd system's algorithmic behavior (e.g., instructional policies learned from data).

Artificial intelligence should be used to provide children with one-to-one tutoring to improve their learning and monitor their well-being, academics have argued. One-to-one tutoring has long been thought the most-effective approach to teaching but would be too expensive to provide for all students. However, in a paper, academics from University College London's Knowledge Lab argue that AI systems could simulate human one-to-one tutoring by delivering learning activities tailored to a student's needs and providing targeted and timely feedback, all without an individual teacher present. Instead of being examined in traditional ways, children could be assessed in a more complete manner by collecting data about their performance over a long period, providing employers and educational institutions with a richer picture of their abilities. The report argues that AI could radically transform our education system for the better – but it is being held back by funding.